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(1) Real Party in Interest 

A statement identifying the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

A statement identifying the related appeals and interferences which will directly affect or 
be directly affected by or have a bearing on the decision in the pending appeal is contained in the 
brief. 

(3) Status of Claims 

The statement of the status of the claims contained in the brief is correct. 

(4) Status of Amendments After Final 

No amendment after final has been filed. 

(5) Summary of Invention 

The summary of invention contained in the brief is correct. 

(6) Issues 

The appellant's statement of the issues in the brief is correct. 

(7) Grouping of Claims 

The rejection of claims 1-22 stand or fall together because appellant's brief does not 
include a statement that this grouping of claims does not stand or fall together and reasons in 
support thereof. See 37 CFR 1.192(c)(7). 

(8) Claims Appealed 

The copy of the appealed claims contained in the Appendix to the brief is correct. 
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(9) 



Prior Art of Record 



5,486,853 



BAXTER et al. 



1-1996 



5,301,344 



KOLCHINSKY 



4-1994 



5,754,227 



FUKUOKA 



5-1998 



(10) Grounds of Rejection 

The following grounds of rejection are applicable to the appealed claims and were set 
forth in a prior Office Action, mailed on 12 May 2004. [Reproduced here for convenience.] 
5. Claims 1-4, 7-10, 11-14, and 17-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 5,486,853 by Baxter et al. ("Baxter") and U.S. Patent 5,301,344 by 
Kolchinsky. 

Regarding claims 1,11 and 21, Baxter discloses an image processing system (figures 8 
and 9), an image processing method of the system comprising: 

executing digital image processing of interval of active pixel by processor 66 (figure 8) to 
perform various functions on pixel data (column 7, lines 18-31: automatic gain control, 
luminance derivation, etc.); 

executing digital control pre- or post-processing according to commands issued during an 
interval of non-active pixel (i.e. during blanking periods); [Column 7, lines 48-55: control 
commands are received by processor 70 during blanking period and executed accordingly.] 

executing digital image processing again (each field of an image constitutes a different 
active/non-active region, so digital image processing is repeated for every field in order to 
process an entire image). 
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Baxter is silent to utilizing an FPGA for executing said image and control processing 
wherein first and second internal logic descriptions, corresponding to each processing, are 
written to the FPGA. Instead, Baxter teaches utilizing dedicated processors 66 and 70 for 
executing each type processing. 

Kolchinsky discloses a reconfigurable image processing system (figure 2) that is 
implemented by FPGAs (22 and 26, figure 2), wherein arithmetic unit 26 is operative to process 
image data. Kolchinsky teaches that, conventionally, separate image processing operations 
require separate hardware (column 1, lines 23-24). Baxter, as noted above, requires separate 
processors (66, figure 8, and 70, figure 9) for image processing and control processing. 

Kolchinsky' s system uses reconfigurable gate arrays to perform a variety of operations, 
so that processing algorithms "can be changed easily and quickly without hardware replacement" 
(column 2, lines 1-2). That is, the image processing functions of multiple dedicated processors 
such as those taught by Baxter are be performed by a single reconfigurable FPGA (arithmetic 
unit 26, figure 2) disclosed by Kolchinsky. 

With reference to figure 3, first, the command corresponding to the operation to be 
executed is read from the command file at step 50. Then, the code corresponding to the operation 
is identified and placed into a register at step 52. Then, the command is executed at step 58 on 
the condition that proper reconfiguration of internal logic has occurred. 

Kolchinsky teaches (figure 4) that both image processing (e.g. image compression, color 
processing) and control processing (e.g. zooming/panning) are executed by the arithmetic unit 26 
(figure 2). 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
to replace Baxter's separate processors 66 and 70 by Kolchinsky' s reconfigurable FPGAs to 
achieve the claimed invention since Kolchinksy provides a much simpler and more hardware- 
efficient system for effecting image and control processing. As explained above, Kolchinsky 
discloses all operations necessary to perform said digital image processing and said digital 
control processing are performed in a single field programmable gate array — arithmetic unit 26. 

Regarding claims 2 and 12, Baxter teaches there is provided an image pick-up element 
(CCD 22, figure 9), the system executing color signal processing during active pixel interval and 
control processing during non-active interval, as addressed above for claim 1 . 

Regarding claims 3 and 13, Baxter teaches interval of non-active pixel is a VBI (column 
7, lines 48-51). 

Regarding claims 4, 7, 14, and 17, Baxter does not expressly disclose utilizing the HBI 
and optical black pixel interval as non-active regions, however, effecting control processing 
during the HBI and optical black pixel intervals was well-known by those skilled in the art and 
was a common practice at the time the invention was made. 

Regarding claims 8-10 and 18-20 Baxter (column 7, lines 18-31) and Kolchinsky (figure 
4) disclose performing white balance, AF, and lightness control processing. 

Regarding claim 22, Baxter's image processing and control pre- or post-processing occur 
in one frame (e.g. control signals for AGC are transmitted during the vertical blanking period for 
performing automatic gain control of the active region of the frame). 
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6. Claims 5, 6, 15, and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Baxter and Kolchinsky, and further in view of U.S. Patent 5,754,227 by Fukuoka. 

Regarding claims 5, 6, 15, and 16, Baxter (column 7, lines 40-43) and Kolchinsky (figure 
4) teach executing compression but do not expressly disclose executing control processing, such 
as code quantity control, in relation to the image compression in the non-active interval. 

Fukuoka discloses a camera interface similar to that of Baxter wherein control commands 
are issued during the non-active interval (column 8, lines 40-43). Fukuoka also teaches 
performing compression on the active part of the image and teaches that control commands sent 
during the non-active interval comprise compression parameters, such as a scale factor, or code 
quantity (column 10, lines 7-19). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Baxter and Kolchinsky by Fukuoka to achieve the claimed invention since the ability 
to adjust the compression ratios and scale factors, as taught by Fukuoka, provides control over 
the compression operations. 
(11) Response to Argument 

INTERPRETATION OF "INTERVAL OF ACTIVE PIXEL" AND "INTERVAL OF NON- 
ACTIVE PIXEL" 

Video signals conventionally consist of an "active" interval and a "non-active" interval. 
The active interval constitutes the actual image data, whereas the non-active interval constitutes 
those portions of the signal that contain blanking intervals. The blanking intervals that appear at 
the end of each line of video data are called the horizontal blanking intervals (HBI). The purpose 
of the HBI is to provide an indication that the video data has reached the end of a line. A video 
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signal is simply a stream of data, so without the presence of the HBIs, one would not know 
where one line of image data ends and where the other begins. Similarly, the vertical blanking 
interval (VBI) appears at the end of each image frame in order to indicate the ending of one 
frame and the beginning of another. 
EXPLANATION OF BAXTER 

Figures 8 and 9 of Baxter illustrate a camera system. The system includes two processors, 
66 and 70. Processor 66 executes operations that correspond to the claimed "digital image 
processing" - e.g. automatic gain control, luminance derivation, color correction, etc. - see 
column 7, lines 18-31. These operations of processor 66 are considered to be executed in the 
"interval of active pixel," because the operations are performed on the actual image data. 

In contrast, processor 70 executes "digital control processing" in the "interval of non- 
active pixel." In column 7, lines 48-61, Baxter discloses that the processor 70 receives control 
signals during VBI for performing control operations such as focusing, zooming, panning, and 
tilting. These control operations are considered to correspond to the claimed "digital control 
processing" because they control the operation of the camera rather than process the actual image 
data. 

Thus, Baxter utilizes processor 66 to execute "digital image processing" in the "interval 
of active pixel" and processor 70 to execute "digital control processing" in the "interval of non- 
active pixel." 

However, the claimed invention calls for utilizing a field programmable gate array 
(FPGA), which is a reprogrammable processor, to perform both the digital image and digital 
control processing during the active and non-active intervals, respectively. According to the 
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claimed invention, once digital image processing is executed, the "internal logic description" of 
the FPGA is rewritten to a second internal logic description (that is, the FPGA is reprogrammed), 
and digital control processing is then executed. Then, the FPGA is reprogrammed so that it once 
again can perform the digital image processing operations, and digital image processing is again 
executed. 

In other words, the claimed invention calls for programming an FPGA to perform image 
processing on data in the active interval and then executing the image processing. Then, the 
FPGA is reprogrammed to perform control processing during the non-active region, and the 
control processing is then executed. Next, the FGPA is once again reprogrammed to perform 
image processing on data in the active region, and the image processing is executed again. 
APPLICATION OF KOLCHINKSY 

Kolchinsky was relied upon to cure the deficiency in Baxter, who utilizes separate 

processors rather than a single FPGA. As Kolchinsky teaches, 

Conventional data processing systems such as image data 
processors require extensive hardware to perform a series of 
arithmetic logic operations on the data. For example, image data 
requires the operations of image acquisition, spatial filtering, 
temporal filtering, histogram equalization, image display, to name 
just a few. Each such operation requires a separate hardware 
implementation in separate boards or separate gate arrays . Such 
implementations are expensive and are relatively inefficient in the 
sense that while all the necessary hardware is always present it is 
being utilized only a small part of the time during the period when 
its particular operation in the series is called for. ( emphasis added) 

Kolchinsky is describing the exact situation found in Baxter wherein separate processors 

perform separate operations relating to video data, and each processor is only being utilized part 



of the time - either during the active interval or the non-active interval. To combat this problem 
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of "expensive" and "inefficient" implementations, Kolchinsky discloses employing a single 
FPGA to replace multiple processors: "[i]t is a further object to provide such a reconfigurable 
sequential processor which uses the same gate arrays reconfigurable in time to implement 
specific generators and arithmetic logic units" (column 1, lines 56-61). The FPGA is 
reprogrammed by software (column 1, line 55) so its algorithm can be changed easily without 
hardware replacement (column 2, lines 1-2). Also, the FGPA is well-known and commercially 
available (see column 6, lines 51-54). 

As established above in the grounds of rejections, it would have been obvious to modify 
Baxter in view of Kolchinsky to replace Baxter's separate processors with a single 
reprogrammable processor, or FPGA, because Kolchinsky teaches the utilization of a single 
FPGA, in lieu of multiple processors, is more efficient. 
RESPONSE TO ARGUMENTS FOR CLAIMS 1-20 

Appellant argues that the combination of Baxter and Kolchinsky is invalid for claims 1 
and 1 1 because "Kolchinsky does not disclose or suggest using an FPGA to perform digital 
control processing" (Appeal Brief, p. 6). Appellant contends that the "zooming/panning" process 
executed by Kolchinsky' s FPGA is not digital control processing. Instead, Appellant asserts that 
the zooming & panning taught by Kolchinsky relates to "manipulation of the captured image, not 
to control of the system" (Appeal Brief, p. 6). 

Examiner disagrees with the above assertions that (1) zooming & panning does not 
constitute "digital control processing," and (2) Kolchinsky does not disclose an FPGA utilized to 
perform digital control processing. 
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As established above in the grounds for rejection, Kolchinsky discloses a list of 
exemplary functions in figure 4 that are performed by the FPGA. Among the list of functions is 
"zooming/panning," which Appellant asserts is not digital control processing. 

In drafting a patent application, applicants are allowed to be their own lexicographers in 
defining the meaning of terms, provided that the terms are well-defined and their crafted 
meanings do not contradict customary usage. However, Kolchinksy did not provide an explicit 
definition of the terms "zooming" and "panning/ 5 so those terms should be construed in 
accordance with their conventional usage. "Zooming" and "panning" are well-known terms of 
art that refer to operations for controlling the position of the camera lens and the position of the 
camera head, respectively, as expressly taught by Baxter (column 7, lines 51-61). There is no 
evidence to suggest that Kolchinsky' s usage of "zooming" and "panning" differs from this 
conventional usage. Therefore, zooming & panning constitutes "digital control processing," and 
Kolchinsky is considered to expressly disclose performing "digital control processing" with an 
FPGA. 

Furthermore, Kolchinksy discloses an "image acquisition" function in figure 4. This 
function presumably refers to an operation for controlling the acquisition of an image, such as 
turning a camera on or off. Kolchinsky does not expressly disclose what "image acquisition" 
entails, but it can be implied that this function also refers digital control processing carried out by 
the FPGA. 

Also, regardless of whether Kolchinsky discloses his FPGA performing "digital control 
processing," the combination of Baxter and Kolchinsky should still be valid. Baxter discloses 
performing the necessary digital image and digital control processing, and Kolchinsky was only 
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relied upon to show that it was obvious and advantageous to replace Baxter's separate processors 
with a single FPGA. In general, FPGAs are multi-purpose, reprgorammable processors that are 
capable of performing virtually any operation or algorithm that can be executed by a dedicated 
processor. Even if Kolchinsky did not expressly disclose performing digital image and control 
processing with an FPGA, that omission would not negative the fact FPGAs are capable of 
performing such processing, nor should it render the proposed combination invalid. 
Nevertheless, Kolchinsky unequivocally demonstrates in figure 4 that the disclosed FPGA is 
capable of performing both the claimed digital image and digital control processing. 

In the first two paragraphs on page 7 of the Brief, Appellant argues against the Baxter 
reference alone. However, the rejection was made based on the combination of Baxter and 
Kolchinsky. One cannot show nonobviousness by attacking references individually where the 
rejections are based on combinations of references. See In re Keller , 642 F.2d 413, 208 
USPQ 871 (CCPA 1981); In re Merck & Co.. 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

In the third paragraph on page 7 of the Brief, Appellant argues that the combination of 
Baxter and Kolchinsky is invalid because Baxter's processors 66 and 70 cannot be combined for 
logistical reasons. Appellant does not allege that the proposed combination changes the principle 
of operation of Baxter or render Baxter inoperable for its intended purpose. Rather, Appellant 
merely alleges that, "in order to combine processor 66 and CPU 70, it would be necessary to use 
a wired logic processing device, such as an ASIC (application-specific integrated circuit), instead 
of a general purpose CPU." It is unclear how this allegation, if true, would affect the proposed 
combination. Kolchinsky's FPGA is arguably an application-specific device in that it performs 
specific functions for which it is programmed. As Kolchinsky teaches, FPGAs are versatile 
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devices that can be programmed to perform a variety of specific operations, and there would 
have been a reasonable expectation of success in replacing Baxter's separate processors with 
Kolchinsky's FPGA 

Moreover, the test for obviousness is not whether the features of a secondary reference 
may be bodily incorporated into the structure of the primary reference; nor is it that the claimed 
invention must be expressly suggested in any one or all of the references. Rather, the test is what 
the combined teachings of the references would have suggested to those of ordinary skill in the 
art. See In re Keller. 642 F.2d 413. 208 USPO 871 (CCPA 1981). See also In re Sneed . 710 F.2d 
1544, 1550, 218 USPQ 385, 389 (Fed. Cir. 1983) ("[I]t is not necessary that the inventions of the 
references be physically combinable to render obvious the invention under review."); and In re 
Nievelt, 482 F.2d 965, 179 USPQ 224, 226 (CCPA 1973) ("Combining the teachings of 
references does not involve an ability to combine their specific structures."). 

With regards to claims 2-10 and 12-20, Appellant asserts they should be allowable for the 
same reasons as claims 1 and 11. Examiner asserts claims 2-10 and 12-20 should be rendered 
obvious for the reasons stated above for claims 1 and 11. 
RESPONSE TO ARGUMENTS FOR CLAIMS 21 and 22 

With regards to claims 21 and 22, Appellant asserts they should be allowable for the 
same reasons as claims 1 and 11. Examiner asserts claims 21 and 22 should be rendered obvious 
for the reasons stated above for claims 1 and 11. 
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In addition, Appellant alleges that the "pre-processing or post-processing" limitation is 
not disclosed or suggested by the cited references. Examiner disagrees with this allegation. 

Claim 21 differs from claim 1 primarily in that the "control processing" is further refined 
as control "pre-processing or post-processing relating to said image processing." Examiner does 
not consider this feature to distinguish the claim from the combination of Baxter and Kolchinsky. 
For example, the control processes of zooming and panning disclosed by both Baxter and 
Kolchinsky are "pre-processes related to the image processing," in that Baxter's camera is 
controlled by zooming and panning prior to capturing image data and performing image 
processing thereon - e.g. an operator would want to pan the camera or zoom on an object prior to 
capturing images of the object; the captured images would then be subjected to image processing 
operations, such as compression or color correction. In other words, controlling the camera is a 
pre-process in that it occurs prior to processing captured image data. 



For the above reasons, it is believed that the rejections should be sustained. 
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